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instructions  are  based  upon  voluntary  informed  consent  and  meet  or  exceed 
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THE  PROBLEM 

A  difficult  aspect  of  predictiag  fleet  pilot  performance  is  acquiring 
meaningful  and  reliable^  inflight  criteria.  Without  such  criteria^ 
performance  assessment  is  both  theoretically  and  realistically  impossible. 
This  study  was  an  attempt  to  predict  Air  Combat  Maneuvering  (ACM) 
performance  using  performance-based  laboratory  tests  and  to  evaluate  the  VF- 
43  adversary  squadron's  grading  of  inflight  ACM  performance  in  the  Fleet 
Fighter  ACM  Readiness  Program  at  Naval  Air  Station  Oceana.  The  purpose  of 
the  latter  effort  was  to  select  convenient  and  reliable  criteria  for  ACM 
performance  assessment  and  use  in  the  validation  of  the  laboratory  tests. 

FINDINGS 

In  an  Initial  evaluation  (Study  I)»  F-4  pilots  performed  in  Fleet 
Fighter  ACM  Readiness  exercises  and  completed  performance-based  perceptual 
motor  and  multitaslc  tests.  Results  indicated  that  dichotic  listening  test 
measures!  obtained  during  multitaslc  conditions,  could  be  used  to  reliably 
predict  ACM  inflight  criteria.  Results  of  a  larger  sample  of  F-14  pilots 
(Study  II)  indicated  that  an  overall  ACM  grade  (OAG)  assigned  by  VF-43 
adversary  personnel  can  be  predicted  reliably  by  an  objective  kill 
difference  composite  score  and  three  subjective  measures:  situational 
awareness,  mutual  support,  and  energy  management.  These  four  measures 
accounted  for  78%  of  the  variance  with  the  OAG.  A  correlational  analysis 
suggests  that  the  VF-43  grading  process  is  reliable  and  consistent. 

Additional  results  were  obtained  on  the  relation  between  the  Naval 
Aerospace  Medical  Research  Laboratory  vision  tests  and  ACM  criteria  (Study 
111).  Contrast  sensitivity  measures  were  significantly  related  to  a  mean 
time-to-£irst-kill  measure^'  Visual  acuity  and  accommodative  flexibility 
measures  were  significantly  related  to  the  Initial  sighting  (tally-ho)  and 
visual  identification  (VID)  of  adversary  aircraft  on  an  instrumented  range. 
Age  and/or  experience  in  ACM  may  be  an  important  variable  in  relating  vision 
tests  to  pilot  performance. 

RECOMMENDATIONS 

1.  Improved  performance-based  tests  should  be  administered  to  a  sample  of 
Navy  pilots  performing  in  Fleet  Fighter  ACM  Readiness  Evaluations  to 
replicate  initial  test  results. 

2.  An  overall  ACM  grade  regression  equation  should  be  applied  to  a 
supplementary  sample  of  pilots  performing  in  Fleet  Fighter  ACM  Readiness 
exercises  to  confirm  the  reliability  and  validity  of  the  VF-43  adversary 
squadron's  grading  process. 
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INTRODUCTIOM 


Research  la  ongoing  at  the  Naval  Aerospace  Medical  Research  Laboratory 
to  predict  fleet  aviator  Inflight  performance  using  perceptual  psychomotor 
and  Information  processing  tasks.  The  goal  is  to  develop  relevant 
laboratory  tasks*  test  aircrew*  and  relate  aircrew  test  performance  to 
simulated  and  actual  flight  performance.  From  this  effort*  It  may  be 
possible  to  aid  decisions  concerning  aircrew  selection,  training  pipeline 
assignment*  and  post-training  aircraft  asslgnmmt.  Crucial  to  this  research 
Is  the  Identification  of  useful*  valid*  and  reliable  measures  of  flight 
performance  for  the  validity  assessment  of  predictor  tests. 

Previous  United  States  Navy  research  to  predict  operational  performance 
has  been  encouraging  (1-7).  Rlckus  and  Berkshire  (4)  reported  that  peer 
ratings  obtained  In  Navy  preflight  training  were  useful  In  Identifying 
successful  and  unsuccessful  aviators  In  combat  (Viet  Nam).  Bale  et  al.  (2) 
evaluated  F-4  Replacement  Air  Group  (RAG)  training  during  the  mld-60s  and 
developed  a  prediction  equation  that  could  reduce  RAG  attrition  from  13.3  to 
8.3%.  A  study  of  Tactical  Aircrew  Combat  Training  System  (TACTS)  F-4  air 
combat  maneuvering  by  Clavarelll  et  al.  (1)  in  the  late  7 Os  found  three 
measures  (angle-off- tall*  closing  velocity*  and  indicated  air  speed)  that 
were  slgnlflcartly  related  to  ACM  performance.  Brlccson  et  al.  (3)  were 
able  to  successfully  predict  F-4  carrier  landing  performance.  Shannon  et 
al.  (7)  found  tliat  a  relatively  small  set  of  RAG  measures  can  reliably 
predict  final  overall  RAG  grade  (multiple  ^  ■  .84).  The  two  most  important 
measures  (carrier  qualification  power/nose'^control  and  offensive  ACM) 
accounted  for  73%  of  the  variance  with  the  final  overall  RAG  grade.  In  two 
subsequent  studies*  Shannon  and  Waag  (6)  found  that  an  equation  based  on  an 
east  coast  RAG  reliably  predicted  performance  of  F-4  pilots  on  a  west  coast 
RAG  and  reported  (S)  that  experience  and  seven  undergraduate  training  grades 
reliably  predicted  final  overall  RAG  grade  (multiple  R  ■■  .51). 

Despite  these  positive  results*  new  aircraft  and  weapons  system 
technology  may  have  made  the  research  obsolete.  In  addition*  the  approaches 
of  the  previous  f  .udles  and  the  present  effort  differ.  Previous  studies  (2- 
6)  used  pencll-and-paper  selection  tests  and  undergraduate  training  measures 
to  predict  performance.  Our  approach  used  performance-based  tests  of 
cognitive*  perceptual*  and  multitask  functioning  to  predict  fleet 
operational  aviator  performance. 

The  present  study  represents  an  attempt  to  predict  ACM  inflight 
performance  using  performance-based  automated  tests*  and  an  evaluation  of 
the  VF-43  adversary  squadron's  grading  of  aircrew  performance  in  the  Fl^et 
Fighter  ACM  Readiness  Program  at  Naval  Air  Station  Oceana.  The  purpose  of 
the  latter  effort  was  to  select  useful  and  reliable  criteria  for  ACM 
performance  assessment  and  validating  future  laboratory  tests.  In  addition* 
correlations  between  measures  of  the  TACTS  and  vision  tests  were  determined. 

Study  I.  Multitask  Test  and  Marine  Pilot  ACM  Performance 

The  purpose  of  this  evaluation  was  to  test  the  feasibility  of 
predicting  ACM  performance  with  performance-based  perceptual-motor  and 
cognitive  multitask  tests. 


SUBJECTS 

Tv7*tity*’tuo  F-4  pilots  from  Marine  Squadron  451  served  as  subjects 
during  their  participation  In  a  rieet  Fighter  ACM  Readiness  Program  exercise 
at  NAS  Oceana  during  the  summer  of  1985. 

PROCEDURE 


Air  combat  maneuvering  performance  data  are  routinely  collected  at  the 
NAS  Oceana  TACTS  facility.  The  data  are  used  by  VF-43  adversary  squadron 
personnel  to  develop  aircrew  and  squadron  ACM  performance  ratings.  The 
performance  ratings  provide  training  feedback  to  Individual  aircrews  by 
highlighting,  their  strengths  and  weaknesses  in  ACM,  and  also  provide  a 
method  for  military  managers  to  assess  overall  squadron  readiness.  In 
addition,  the  TACTS  Fleet  Fighter  ACM  Readiness  Exercise  results  serve  as  a 
base  to  evaluate  the  tactical  employment  of  aircraft  and  weapon  systems. 
Typical  data  resulting  from  the  readiness  exercises  are  presented  In  Table 
1.  A  description  of  the  TACTS  training  system  and  definitions  for  specific 
TACTS  performance  measures  are  In  Appendixes  A  and  B,  respectively. 


Eighteen  of  the  22  Marine  pilots  completed  single-  and  multitask 
cognitive  and  perceptu»l-motoc  teats  during  the  readiness  exercise.  The 
tests  consisted  of  a  24-tvtal  dichotlc  listening  task  (DLT)  followed  by  6- 
fflln  performance  on  a  psychomotor  task.  Both  tasks  were  th^'i  performed 
simultaneously.  Correlational  and  multiple  regression  analyses  were 
conducted  on  the  ACM  performance  measures  to  Identify  suitable  criteria  and 
to  evaluate  the  strengths  of  the  correlations  among  the  various  measures. 
Subjects'  test  performance  data  were  correlated  with  the  Identified 
criteria. 

RESULTS 

Pearson  correlations  of  27  measures  associated  with  the  VF-43  adversary 
squadron's  evaluation  of  ACM  performance  of  22  F-4  pilots  are  presented  in 
Table  1.  The  overall  ACM  grade  (OAG)  was  significantly  related  to  offensive 
maneuvering  (£  ■■  .67),  situational  awareness  <£  ••  .81),  and  mutual  support 

(r^  "  .56).  In  addition,  the  OAG  was  significantly  related  to  mean  time  to 
first  kill  (£  ■  -.42),  number  of  VF-43  adversary  squadron  miscile  shots  (£  ■■ 
-.65),  and  the  number  of  timet  a  pilot  was  "killed"  (r  ■■  -.70)  In  the 
simulated  exercises.  The  negative  correlations  Indicated  that  a  higher  ACM 
grade  was  associated  with  shorter  times  to  first  kill,  fewer  adversary 
squadron  missile  shots  taken,  and  fewer  times  being  "killed"  In  the 
simulated  exercises  (better  ACM  performance). 


A  multiple  regression  analysis  indicated  that  situational  awareness, 
offensive  maneuvering,  number  of  times  killed,  and  mutual  support  accounted 
for  89%  of  the  variance  associated  with  the  OAG  criterion  (R  .95,  £(4,  17) 
»  36.19,  2^  <  .0001).  The  OAG  and  these  four  measures  were  th^n  correlated 
with  the  single-  and  multitask  cognitive  end  psychomotor  test  performance  of 
18  of  the  22  pilots  .  A  derived  composite  kill-difference  score  (the  total 
number  of  ACM  kills  minus  the  number  of  times  a  pilot  was  killed  in  the 
TACTS  simulated  exercises)  was  Included  In  the  correlational  analyses  as 
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Four  pilots  did  not  volunteer  to  complete  the  series  of  tests. 
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wftll  as  the  total  aumber  of  flight  hours,  which  ranged  from  337  to  1923.  Of 
the  42  correlations  computed  between  the  tests  and  ACM  criteria,  4  (10%) 
were  significant  at  the  .05  or  .01  level  of  confidence. 


TABLE  1.  Pearson  Correlations  of  Individual 
ACM  Performance  Measures  with 
Overall  ACM  Grade(a)  (£  22). 


Subjective  measures 

_r 

Use  of  environment 

.05 

Start/VID  start 

.07 

First  move 

-.18 

Aggressiveness 

.16 

Offensive  maneuvering 

.67** 

Defensive  maneuvering 

-.04 

Keeping  sight/ lookout 

.39 

Energy  management 

.23 

Mental  plot 

-.17 

Situational  awareness 

.31** 

Bugout  technique 

-.13 

Weapon  system  employment 

.24 

VXD  technique 

-.08 

VXD  communication 

.05 

UHF  communication 

.24 

Game  plan  usage 

.16 

Mutual  support 

.56** 

Debrief 

-.17 

Recons truction 

-.37 

Objective  Measures 

_r 

Total  number  TACTS  kills 

.31 

Number  of  missiles  launched 

.16 

Mean  time  (s)  to  first  kill 

-.42* 

Visual  tally-ho  mean  range 

.01 

VXD  mean  range 

-.05 

Number  of  times  killed 

-.70** 

Number  VF-43  missile  shots 

*  £  <  .05 
**  £  <  .01 

-.65** 

(a)  The  overall  grade  is  a 
subjective  measures. 

composite  of  the  19 

Note:  Performance  measure  definitions  are  in 
appendix  B. 

The  Pearson  correlations  of  the  cognitive  and  psychomotoi:  tests  shown 
in  Table  2  indicated  that  a  DLT  measure  (DLT-l)  obtained  during  multitask 
performance  was  significantly  related  to  offensive  maneuvering  (r  =  .62)  and 
the  kill-difference  composite  score  (r  =*  .49).  A  DLT  multitask  measure 


based  oa  a  slightly  different  scoring  procedure  (DLT-2)  was  significantly 
related  to  the  OAG  (r^  >  .49)  and  the  offensive  meneu/ering  score  (£  -  .60). 
Number  of  flight  hours  was  unrelated  to  any  ACM  or  test  performanc^measure. 
These  results,  although  based  on  a  small  sample  of  Marine  pilots,  support 
the  feasibility  of  developing  a  series  of  performance-based  cognitive, 
perceptual,  and  multitask  tests  to  predict  aviator  performance. 

TABLE  2.  Correlation  of  ACM  Performance  Criteria  and  Single  and  Multitask 
Cognitive  and  Psychomotor  Tests  <£  -  18  F-4  Pilots). 


Measures 


1.  Overall  ACM  grade 

2.  Situational  auareness  .83** 

3.  Offensive  msneuverlng  .83**  .61** 

4.  Number  times  killed  -.75**  -.58*  -.58* 


5.  Mutual  s^>port 

6.  Kill-difference  score 

7.  Fli^t  hours 

.51* 

.50* 

.13 

.58* 

.25 

-.01 

.25 

.41 

.06 

-.21 

-.59** 

-.39 

-.03 

.21 

.05 

a.  MulUta^  IF.T  1 

.43 

.10 

.62** 

-.27 

-.16 

.49* 

.11  1 

9.  Multitask  DLT  2 

.49* 

.12 

.60** 

-.33 

-.03 

.37 

.27  1 

.93** 

10.  Multitask  RiT 

-.14 

.04 

-.25 

.u 

.20 

-.05 

-.18  1 

-.40 

-.45 

11.  Single  task  DLT  1 

.21 

-.02 

.28 

-.17 

-.15 

.35 

.04  1 

.56* 

.53* 

-.72** 

12.  Single  task  DLT  2 

.32 

.15 

.36 

-.20 

-.19 

.22 

-.10  1 

.60** 

.55** 

-.67**  .85** 

13.  Single  task  RIT 

.10 

-.03 

.16 

-.18 

-.29 

.16 

.07  1 

.27 

.29 

.31  -.12 

►  5 

6  : 

1  8 

9 

10 

11 

12  ^ 

iij 

*.C5  -  .47 
**.01  -  .59 


Unresolved  was  a  series  of  important  questions  concerning  the  Fleet 
Fighter  ACM  Readiness  Program  evaluation  process:  Are  the  resulting  grades 
reliable  for  Navy  pilots  flying  contemporary  F-14  aircraft?  What  is  the 
relation  of  the  VF-43  grading  process  to  more  objective  TACTS  ACM 
performance  measures  (l,e.,  total  number  of  kills,  visual  identification 
(VID)  range,  VID  kills  and  engaged  kills)?  Which  ACM  measures  are  most 
predictive  of  ACM  performance? 

Stedy  llj  Fleet  Fighter  ACM  Readiness  Program  Grades  as  Criteria 

The  purpose  of  the  second  study  was  to  answer  the  questions  posed 
above,  and  assess  the  utility  of  VF-43  ACM  grades  and  TACTS  ACM  performance 
measures  as  criteria  for  the  validation  of  tests  developed  to  predict  ACM 
performance. 


SUBJECTS 

Subjects  were  125  Navy  F-14  pilots  (10  fighter  squadrons)  who 
participated  in  the  Fleet  Fighter  ACM  Readiness  Program  against  the  VF-43 
adversary  squadron  at  NAS  Oceana  during  1985  and  January  of  1986. 
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PROCEDURE 

Air  combat  maneuvering  "competitive  exercise"  performance  data  were 
collected  for  the  Navy  participants  in  the  Fleet  Fighter  ACM  Readiness 
Program  and  analyzed  to  derive  correlational  statistics.  Multiple 
regression  analyses  were  performed  to  study  the  relative  importance  of 
specific  predictors  and  to  derive  criteria  that  would  predict  the  overall 
ACM  grade. 

RESULTS 

Performance  measure  descriptive  statistics  and  Pearson  correlations 
between  the  OAG  and  19  subjective  and  12  objective  TACTS  measures  are 
presented  in  Table  3.  The  kill-difference  composite  score  was  the  measure 
most  highly  correlated  with  OAG  <£  -  ,76)^  followed  by  the  engaged  kill- 
difference  composite  score  (£  ■  .70).  The  total  number-of-kills  measure  was 
related  to  OAG  (£  -  .65),  as  were  missiles  launched  <£  -  .58),  VID-kill  <£ 

■■  .39)  and  engaged-kill  (£  ■>  .57)  scores.  As  expected,  the  number-of-times- 
killed  measure  was  significantly  and  negatively  related  to  OAG  (r  -.51). 
Number  of  radar  locks  was  significantly  correlated  with  the  OAG  Tr  »  .24)  as 
well.  Four  of  the  objective  measures--mean  time- to-f irst-klll,  radar  lock 
mean  range,  visual  tally-ho  mean  range,  and  VXD  mean  range  measures--were  not 
significantly  tel  \  to  OAG. 

We  werr  sur); .  <$8  that  the  mean  time- to-f irst-hill  score  was  unrelated 

bo  OAG.  One  expla  Ion  for  this  result  may  be  that  the  time- to-f irst-kill 
score  is  an  average  of  both  VXD  and  engaged-kill  times.  This  pooling  of 
relatively  short  (VXD)  and  longer  (engaged)  kill  times  may  have  a 
confounding  ef/ject  on  the  resulting  correlations.  Separation  of  VXD  and 
engaged  kill  times  might  enable  a  better  understanding  of  the  relation  of 
this  ACM  score  with  the  OAG  and  other  TACTS  measures. 

An  examination  of  the  subjective  measures,  as  shown  in  Table  3, 

Indicated  that  11  measures  were  significantly  correlated  with  the  OAG. 
Situational  awareness  (described  by  VF-43  adversary  personnel  as  a  synonym 
for  ACM  proficiency)  correlated  most  highly  with  OAG  (£  >■  .70),  followed  by 
offensive  maneuvering  (£  “  .53),  aggressiveness  (£  ••  .45),  mutual  support  (£ 
■  .44),  and  start/VXO  start  (£  “  .40).  Defensive  maneuvering,  keeping 
sight,  energy  management,  weapon  system  employment,  VXD  technique,  and  game 
plan  measures  were  significantly  correlated  with  Che  OAG  as  well,  with 
correlations  between  .23  and  .39.  Those  measures  not  significantly 
correlated  with  the  OAG  were  use  of  environment,  first  move,  mental  plot, 
bugout  technique,  VXD  communication,  UHF  communication,  debrief,  and 
recons  true  tion . 

The  full  correlation  matrix  is  presented  in  Appendix  C.  Of  496 
correlations,  163  (33%)  were  significant  at  the  .05  level  or  above. 


TABLE  3.  Performaace  Measure  Descriptive  Statistics  and  Pearson  Correlations 
Between  Overall  ACM  Grade  and  ACM  Performance  Measures  ~  125). 


Subjective  measures 

jf 

Mean 

SD 

Min 

Max 

Overall  grade 

— 

2.01 

.05 

1.90 

2.15 

Use  of  environment 

.00 

2.01 

.05 

1.83 

2.20 

Start/VID  start 

.40** 

2.04 

.16 

1.67 

2,88 

Fitst  move  (n  -  113) 

.12 

2.01 

.18 

1.50 

2.50 

Aggressiveness 

.45** 

2.08 

.13 

1.88 

2.75 

Offensive  maneuvering 

.53** 

2.08 

.18 

1.67 

2.50 

Defensive  maneuvering 

.39** 

1.96 

.14 

1.56 

2.25 

Keeping  sight/lookout 

.35** 

1.95 

.14 

1.50 

2.29 

Energy  management 

.37** 

2.00 

.15 

1.57 

2.38 

Mental  plot 

-.11 

1.98 

.11 

1.50 

2.90 

Situational  awareness 

.70** 

1.93 

.25 

1,25 

2,75 

Bugout  technique 

.11 

2.03 

.17 

1.67 

2.50 

Weapon  system  employment 

.30** 

2.06 

.19 

1.25 

2.50 

VXD  technique 

.30** 

1.99 

.12 

1.67 

2.33 

VXD  communication 

-.08 

1.99 

.05 

1.75 

2.17 

UHF  communication 

.12 

2.00 

.08 

1.50 

2.25 

Game  plan  usage 

.32** 

2.08 

.18 

1.50 

2.50 

Mutual  support 

.44** 

1.95 

.26 

1.33 

2.50 

Debrief  (n  >  112) 

.11 

2.01 

.07 

2.00 

2.50 

Reconstruction 

.16 

2.01 

.04 

2.00 

2.25 

Objective  measures 

£ 

Mean 

SD 

Min 

Max 

Total  number  TACTS  kills 

.65** 

5.63 

2.80 

0.0 

14.0 

Number  of  missiles  launched 

.58** 

13.14 

6.85 

1.0 

31,0 

Number  of  VXD  kills 

.39** 

2.66 

1.64 

0.0 

7.0 

Number  of  engaged  kills 

.57** 

2.97 

2.05 

0.0 

10.0 

Mean  time  (s)  to  first  kill 

-.02 

42.98 

25.31 

6.0 

115.3 

(n  -124) 

Number  of  radar  locks 

.24** 

4.95 

1.55 

0.0 

7.0 

Radar  locks  mean  range 

-.04 

13.82 

3.22 

0,0 

26.0 

Visual  tally-ho  mean  range 

.12 

2.80 

1.37 

0.0 

6.5 

VID  mean  range 

.16 

1.59 

0.88 

0,0 

5.2 

Number  of  times  killed 

-.51** 

1.68 

1.10 

0.0 

5.0 

Kill-difference  score 

.76** 

3.92 

3.14 

-2.0 

13,0 

Engaged-kill-difference  score 

.70** 

1.29 

2.44 

-3.0 

9.0 

**  £  <  .01 


REGRESSION  ANALYSIS 

To  examine  which  subjective  and  objective  measures  would  best  predict 
the  OAG,  a  series  of  forward  selection  multiple  regression  analyses  (8)  was 
conducted.  A  forward  selection  stepwise  multiple  regression  technique  was 
used  because  of  multi-collnearity  (high  intercorrelations)  among  certain  of 
the  objective  and  subjective  measures.  The  first  regression  (Appendix  D, 
Table  D-1)  was  based  on  the  subjective  measures  in  Table  3.  Results 
indicated  that  a  6-measure  regression  model  accounted  for  83%  of  the 
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vaxiaace  with  OAG  “  ,91,  JF(6,  113)  -  98,57,  ^  <  .0001).  The  situational 
awarenaaa  measure  entered  the  regression  first  and  accounted  for  49%  of  the 
variance  with  the  OAG.  Offensive  maneuvering,  mutual  support,  start/VID 
start,  energy  management,  and  keeping  sight  measures  then  entered  the 
regression  equation  accounting  for  11,  7,  S,  4,  and  4%  additional  variance, 
respectively. 

A  second  regression  analysis  (Appendix  D,  Table  D-2)  was  conducted  using 
the  objective  performance  measures  in  Table  3.  The  measures  total  TACTS 
kills-  kill-difference  score,  and  engaged  kill-difference  score  were 
excluded  because  they  represented  combinations  of  other  measures.  As  shown 
in  Appendix  C,  number  of  missiles  launched  was  related  to  total  TACTS  kills 
(jf  -  .83),  engaged-kills  (£  ■  .67).  VID-kills  (r  ■  .57),  and  the  kill- 
dTffcrence  score  (r  ■■  .78).*~  Since  this  measure  Is  simply  a  means  of 
achieving  TACTS  kills,  it  too  was  excluded.  These  composite  and/or 
duplicative  measures  were  omitted  from  the  regression  to  increase  insight 
into  those  specific  performance  measures  most  important  to  the  OAG.  The 
resulte  of  the  multiple  regression  indicated  that  engaged-kills,  number-of- 
times-kllled,  and  VlD^kllls  accounted  for  62%  of  the  variance  with  the  OAG. 
The  engaged-kill  measure  entered  the  model  first  and  accounted  for  33%  of  the 
variance  associated  with  the  OAG.  Number-of- timea-killed  and  VID-kill 
measures  followed  in  succession,  accounting  for  19  and  10%  additional 
variance,  respectively.  Finally,  the  mean  time- to-flrst-klll  measure  entered 
the  regression  model  but  accounted  for  only  1%  additional  variance  ■■  .79, 
F^(4,  120)  <"  51.02,  £  <  .0001).  It  ia  important  that  both  engaged-kills  and 
VlU-kills  entered  the  regreaslon  model  (both  are  significantly  correlated 
with  OAG,  but  the  correlation  between  the  two  measures  is  low,  r  *■  *14). 

Theae  results  suggest  that  the  VID-kill  and  engaged-kill  performlance 
measures  are  statistically  independent  in  this  population  of  Navy  pilots. 

They  also  emphasize  the  Importance  of  pilot  training  in  both  of  these  ACM 
skills. 

A  third  multiple  regression  model  (Appendix  D,  Table  D-3)  was  based  on 
a  kill-difference  score  (o  composite  of  the  first  three  measures  entering 
ths  second  regression  model)  and  the  subjective  measures  of  Table  3.  The 
kii-l-dlfference  measure  enterec  the  regression  first,  accounting  for  57%  of 
the  variance  with  GAG.  Next,  the  situational  awareness  measure  entered  the 
regression,  accounting  for  an  additional  14%  of  variance,  followed  by  energy 
management  and  mutual  support,  which  each  contributed  about  4%  additional 
variance  (R  -  .89,  ^(4,  120)  -  109.39,  £  <  .0001). 

These  resulcs,  indicating  the  Importance  of  kills  in  the  VF-43 
adversary  squadrou's  grading  of  ACM  performance,  were  expected  since  kill 
ratios  from  the  competitlvo  exercises  of  the  Fleet  Fighter  Readiness  Program 
represent  a  basic  component  of  the  gi'cding  process  (9).  Unexpectedly, 
situational  awareness  and  other  subjective  measures  contribute  Important 
additional  variance  in  the  prediction  of  CAG,  Apparently  human  judgement  of 
ACM  proficiency  is  an  important  element  la  these  performance  evaluations. 

SITUATIONAL  AWARENESS 

At  NAS  Oceana,  VF-43  adversary  personnel  define  situational  awareness  as 
the  "total  of  ACM."  This  definition  se^ms  appropriate  based  on  the  results 
reported  here.  Table  3  shows  that  situational  awareness  is  the  subjective 
measure  most  strongly  related  to  OAG  (r  =  .70).  Item  11  in  Appendix  C  shows 
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the  various  correlatloas  between  situational  awareness  and  the  other 
objective  and  subjective  measures. 

Those  measures  most  strongly  related  to  situational  awareness,  in 
addition  to  the  OAG,  are  the  kill-difference  score,  engaged- kill-difference 
score,  number-of- tiraes-killed,  VID-kills,  engaged-kills,  total  TACTS  kills, 
and  number  of  missiles  launched.  Subjective  measures— start/VXD  start, 
aggressiveness,  offensive  maneuvering,  defensive  maneuvering,  keeping  sight, 
VXD  technique,  game  plan  and  mutual  support— are  also  significantly  related 
to  the  situational  awareness  measure. 

Objective  measures  unrelated  to  situational  awareness  (in  this  analysis) 
are  visual  tally  and  VXD  range,  number  of  radar  locks,  radar  lock  range,  and 
mean  time-to-flrst-kill.  Subjective  measures  that  are  not  significantly 
related  to  i^ituational  awareness  are  environment,  f^rst  move,  energy 
management,  mental  plot,  bugout,  weapon  employment,  VXD  and  UHF 
communication,  debrief,  and  reconstruction. 

PERFORMANCE  MEASURE  RELIABILITY 

An  important  aspect  of  this  study  concerns  the  reliability  or 
consistency  of  the  VF-A3  performance  measures.  To  evaluate  the  reliability 
of  the  TACTS  objective  performance  scores  and  the  more  subjective  VF-43 
scoring  process,  we  randomly  divided  the  Navy  pilot  sample  in  half  and 
examined  performance  measure  correlations  with  the  OAG  (Table  4).  Subjects 
were  divided  on  the  basis  of  even/uneven  chronological  subject  number  • 

Table  4  includes  Pearson  correlations  based  on  the  total  sample  to  allow 
comparison  with  the  correlations  of  both  subsamples.  In  addition,  the 
absolute  difference  of  the  Pearson  correlations  are  presented.  Table  4 
reveals  remarkably  similar  results,  especially  for  the  more  objective  TACTS 
parameters.  The  one  objective  measure  that  indicated  a  major  correlation 
change  was  the  mean  time-to-flrst-kill  measure,  with  an  £  of  .09  for  the  even 
and  -.12  for  the  uneven  subsample,  an  absolute  difference  of  .21.  This 
change  in  correlational  value  is  not  significant  at  the  .05  level.  Moreover, 
the  mean  time-to-first-kill  measure  is  not  significantly  related  to  the  OAG. 
All  other  objective  measure  correlational  values  were  highly  similar.  Six  of 
the  subjective  measures  had  an  absolute  difference  correlation  value  of  .20 
greater.  Only  two  of  these  measures,  UHF  communication  and  reconstruction, 
represented  a  significant  difference  between  the  even  and  uneven  pilot 
subsamples  (£  <  .05),  based  on  a  Fishers  Z  test  of  significant  differences  of 
correlations.  Neither  of  these  correlational  values,  however,  were 
significantly  correlated  with  OAG  for  the  total  sample  or  the  two  subsamples. 
In  summary,  the  objective  and  subjective  measures  most  highly  correlated  with 
OAG  differ  only  slightly  for  the  two  randomly  derived  samples. 

A  second  approach  to  establishing  the  reliability  of  the  OAG  was  to 
apply  the  regression  model  of  Appendix  D,  Table  D-3  (based  on  the  klll- 
dlfference,  situational  awareness,  energy  management,  and  mutual  support 
measures)  to  various  pilot  subsamples.  This  particular  regression  model  was 
used  because  it  represents  the  best  prediction  of  OAG  using  both  objective 
and  subjective  performance  measures.  A  Pearson  correlation  value  was 


Suoject  performance  data  were  ordered  for  statistical  analysis  by  date 
of  the  ACM  readiness  evaluation  and  the  alphabetical  order  of  pilot  name. 
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computed  between  the  predicted  and  actual  OAG  grade  for  eight  different  pilot 
subsamples  (Table  5).  Based  on  a  Fishers  Z  transformation,  the  average  of  the 
eight  correlation  values  is  .88* 


TABLE  4.  Pearson  Correlations  for  Total,  Even,  and  Uneven  Ordered  Pilots, 
and  Correlation  Absolute  Difference  Scores. 


Performance  measure 
correlation  with  OAG 

All  pilots 
(N  -  125) 

Even 
(£  ■•  62) 

Uneven 
(n  -  63) 

Difference 

Use  of  environment 

.00 

.03 

-.03 

.06 

SUrt/VlD  start 

.40** 

.38** 

,42** 

.04 

First  move 

.12 

.05  (n»56) 

.20  (n-57) 

.15 

Aggressiveness 

.45** 

,41** 

,47** 

.06 

Offensive  maneuvering 

.53** 

.49** 

.57** 

.08 

Defensive  maneuvering 

,39** 

.38** 

.41** 

.03 

Keeping  sight/ lookout 

.35** 

.45** 

.26* 

.19 

Energy  management 

.37** 

.46** 

,28* 

.18 

Mental  plot 

-.11 

-.13 

-.06 

.07 

Situational  awareness 

,70** 

.63** 

.76** 

.13 

Bugout  technique 

.11 

.22 

.01 

.21 

Weapon  system  employment 

.30** 

,39** 

.18 

.21 

VID  technique 

.30** 

.29* 

.34** 

,05 

VXD  communication 

-.08 

.02 

-.16 

.18 

UHF  communication 

.12 

-.15 

,33** 

.48**(1) 

Game  plan  usage 

.32** 

.42** 

.22 

.20 

Mutual  support 

,44** 

,38** 

,49** 

.11 

Debrief 

.11 

.10  (n-55) 

.12  (n-57) 

.02 

Reconstruction 

.16 

,32**  “ 

-.10 

.42*(1) 

Total  number  TACTS  kills 

.65** 

.68** 

.61** 

.07 

Number  missiles  launched 

,58** 

.60** 

.56** 

.05 

Number  of  VID  kills 

,39** 

,33** 

,45** 

.12 

Number  of  engaged  kills 

,57** 

.63** 

.51** 

.12 

Mean  time-to-flrst'-klll 

-.02 

.09  (n-61) 

-.12  (n-63) 

.21 

Number  of  radar  locks 

,24** 

.23 

.26*  ” 

.03 

Radar  locks  mean  range 

-.04 

.02 

-.08 

.10 

Visual  tally-ho  mean  range 

.12 

.04 

.17 

.13 

VID  mean  range 

.16 

.07 

.22 

.15 

Number  of  times  killed 

-,5i** 

-.47** 

-.56** 

.09 

Kill-difference  score 

,76** 

.79** 

.73** 

.06 

Engaged  kill-difference  score 

.70** 

,74** 

,67** 

.07 

*  £  <  .05 
**  £  <  ,01 

(1)  Fisher  Z  test  of  significance  of 

Pearson  correlations 

TABLB  5.  P«ax8on  Correlatioa  Values  for  Eight  Pilot  SubsampLes  Based  on 
Predicted  and  Actual  OAG. 


Pilot  subsamples 

£ 

Number 

Even 

.89 

62 

Uneven 

.88 

63 

First  half 

.84 

62 

Second  half 

.90 

63 

1st,  3rd  quarter 

,88 

62 

2nd,  4th  quarter 

.89 

63 

2nd,  3vd  quarter 

.89 

62 

1st,  4th  quarter 

Average  jr  (based  on  Fisher  Z 
All  values  significant,  £  <  . 

.89 

transformations)  »  .88 
01. 

63 

In  summary,  regardless  of  the  sampling  procedure,  the  model  for 
predicting  the  OAG  provided  similar  results.  Because  the  grading  of  the 
Aircrews  by  different  adversary  pilots  seems  consistent,  we  can  assume  that 
the  iutemal  criteria  by  which  the  grades  are  assigned  are  similar  across 
adversary  pilots.  In  essence,  the  grading  process  appeats  reliable. 

PILOTS,  AIRCREW,  AMD  VISUAL  PERFORMANCE 

Although  the  F-14  aircraft  normally  employs  both  a  pilot  and  a  Radar 
Intercept  Officer  (RIO)  working  as  a  team,  we  addressed  those  measures 
associated  with  pilot  performance.  The  VF>43  scoring  process  emphasizes 
pilot  proficiency,  since  the  pilot  maneuvers  the  aircraft,  fires  the 
missiles,  and  as  the  aircraft  commander  is  responsible  for  engagement 
outcome.  However,  it  is  important  to  realize  that  the  RIO's  efforts  in 
operating  the  radar,  keeping  sight,  and  performing  lookout  have  an  important 
effect  on  ACM  engagement  outcome.  Consequently,  we  included  two  RIO 
measures  (number  of  radar  locks  and  radar  lock  mean  range)  in  this  analysis 
to  examine  the  relation  of  RIO  performance  to  pilot  tally-ho  and  pilot 
aircraft  identification  range  (important  in  pilot  visual  performance).  The 
importance  of  the  RIO's  radar  skills  and  pilot  visual  performance  is 
demonstrated  by  the  significant  Pearson  correlations  between  the  number  of 
radar  locks,  visual  tally-ho  mean  range,  VID  mean  range,  and  other  objective 
TACTS  ACM  scores  of  Table  6. 

Initially,  it  was  unclear  as  to  why  an  RIO  performance  parameter 
(number  of  radar  locks)  would  be  significantly  related  to  pilot  visual 
tally-ho  (£  “  .43)  and  VID  mean  range  ■  ,50).  After  careful 
consideration,  we  believe  that  a  radar  lock  acts  to  decrease  the  visual 
target  search  area  for  the  pilot,  who  then  can  attend  to  the  diamond^, 
knowing  that  an  adversary  aircraft  will  ultimately  become  a  visual  target  at 
the  indicated  location  on  the  head-up  display. 


An  area  of  the  head-up  display,  delineated  as  a  diamond  shape,  indicating 
location  of  the  radar  target. 


the 
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TABLE  6.  Pearson  Correlations  Between  Radar  Lock8»  Pilot  Visual  Tally  and  V£D, 
and  Objective  ACM  Performance  (N^  «  125  Navy  Pilots). 


ACM  objective  measure 

Number  of 
radar  locks 

Radar  lock 
mean  range 

Visual  tally 
mean  range 

VID 

mean  range 

Total  number  TACTS  kills 

,41** 

-.10 

.30** 

.41** 

Number  missiles  launched 

,40** 

-.21* 

,32** 

.42** 

Number  of  VXD  kills 

,42** 

.12 

.45** 

.48** 

Number  of  engaged  kills 

.22* 

-.05 

.05 

.16 

Mean  time- to- £ ire  t-ki 1 1 

-.12 

-.01 

-.19* 

-.14 

Number  of  radar  locks 

- 

.25** 

.43** 

.49** 

Radar  lock  mean  range 

.25** 

- 

.05 

1 

• 

o 

Visual  tally-ho  mean  range 

.43** 

.05 

m 

VXD  mean  range 

.50** 

-.07 

.68** 

- 

Numbet  of  times  killed 

.06 

.00 

.10 

.07 

Kill-difference  score 

.34** 

-.09 

.23** 

,33** 

Engaged-kill-difference  score  .14 

-.05 

.00 

.11 

*  £  <  ,05 

**  £  <  ,01 

A  radar  lock  is  important  to  killSf  and  it  is  a  requirement  for  successful 
use  of  a  forward  aspect  missile.  We  found  significant  correlations  between 
number  of  radar  locks  and  VXD  kills  <£  ■  .42)  and,  to  a  lasser  extent,  engaged 
kills  (£  .22).  The  number  of  radar  locks  was  significantly  related  to  total 

number  of  TACTS  kills  (r  ■  .41),  number  of  missiles  launched  <•  .40),  and  the 
kill ‘•difference  score  (r  *  .34).  Radar  lock  mean  range  was  00^  a  significant 
predictor  for  the  majority  of  objective  ACM  performance  measures.  Apparently, 
when  radar  lock  is  accomplished,  it  occurs  at  distances  so  great  that  the 
variability  in  lock  ranges  does  not  influence  subsequent  ACM  performance. 
Failure  to  acquire  radar  lock  is  another  matter,  however,  as  noted  above. 

Visual  tally'ho  mean  range  and  Vir  mean  range,  as  previously  noted,  are 
strongly  correlated  with  the  radar  loc'..  measure.  Apparently,  a  radar  lock 
significantly  enhances  the  pilot's  acquisition  of  visual  Urgets.  Since  VID 
of  adversary  aircraft  is  required  before  missile  launch,  under  present 
tactical  rules,  vision-dependent  ACM  performance  measures  should  be  positively 
related  to  number  of  TACTS  kills.  Our  data  support  this  hypothesis.  That  is, 
a  greater  visual  tally-ho  range  and  greater  VID  range  are  each  significantly 
associated  with  a  greater  number  of  TACTS  kills  (£  *  .30  and  .41, 
respectively).  Further,  we  hypothesized  that  vision-dependent  ACM  performance 
might^be  more  highly  related  to  the  number  of  VID  kills  rather  than  engaged 
kills  •  Our  data  support  this  hypothesis  also.  Visual  tally-ho  performance 


VID  kills  are  those  that  occur  immediately  following  initial  target 
detection  and  identification  and  are  generally  made  with  radar  directed 
uiitisiles  fired  head-on.  Engaged  kills  occur  during  subsequent  dogfighting, 
when  pilots  attempt  to  maneuver  behind  their  adversary  to  fire  guns  or  heat 
seeking  missiles. 


is  slgnlflcsatly  correlated  with  VXD  kills  >  .45)  but  not  engaged  kills 
-  .05).  Visual  identification  perfcrmance  is  also  significantly  related 
to  VXD  kills  (£  -  .48)  and  not  engaged  kills  (r  -  .16). 


Number  of  radar  lockst  visual  tally-ho  mean  range,  and  VXD  mean  range 
are  negatively  related  to  the  mean  time- to- first-kill  measure.  These 
correlations  are  negative  since  the  launch  of  a  forward  aspect  missile  (the 
best  means  of  achieving  a  quick  kill)  generally  depends  on  achieving  each  of 
'  thesa  measures  in  sequence.  Having  a  longer  tally-ho  or  VXD  range  enables 
‘better  aircrew  preparation  at  the  merge  and  reduces  tlme-to-first-kill. 
Additionally,  a  radar  lock  allows  more  certainty  in  visual  search  and 
produces  longer  range  visual  target  acquisition  and  aircraft  identification. 
In  summary,  the  relation  of  radar  locks  to  Improved  vision-dependent  ACM 
performance  and  the  relation  of  visual  tally-ho  and  VXD  performance  to 
subsequent  missile  launch  and  a  VXD  kill  reflect  a  necessary  sequence  of 
performance  events  for  achieving  mission  success. 

Study  Hit  Thu  Rulation  of  Vlsi<m  Test»  Experleneu,  and  Perfcrmance 

A  series  of  psychophysical  vision  tests  developed  at  the  Naval 
Aerospace  Medical  Research  Laboratory  (NAMRL)  is  currently  being  evaluated 
to  determine  critical  visual  skills  required  in  naval  aviation.  The 
availability  of  both  TACTS  ACM  performance  and  vision  test  data  represented 
an  opportunity  to  study  the  correlational  relation  between  the  two.  In 
addition,  pilot  age  and  flight  hours,  collected  as  a  part  of  the  vision  test 
effort,  enabled  us  to  study  the  relation  of  experience  with  TACTS  ACM 
performance, 

SUBJECTS 

Eighty-nine  of  the  125  Navy  pilots  of  this  evaluation  participated  in  a 
visual  testing  evaluation  at  NAS  Oceana.  Not  all  subjects  completed  all 
tests  in  the  vision  test  battery. 

PROCEDURE 

Relevant  ACM  criteria  (OAG,  situational  awareness,  kill-difference 
score,  VID-kills,  engagod-kills,  and  number  of  times  killed)  of  Study  XX 
were  correlated  with  vision  test  scores.  Additionally,  TACTS  visual  ully- 
hot  and  VXD  range  scores  were  included  because  of  their  relevance  to  pilot 
vision  test  performance.  The  mean  tlrae-to-first-klll  measure  was  Included 
because  it  might  be  related  to  pilot  experience.  Pilot  experience  measures 
were  age  and  flight  hours  (jet  hours,  total  flight  hours,  TACTS  flight 
hours,  and  total  ACM  hours)  as  reported  by  subjects  during  vision  testing. 

RESULTS 

Table  7  presents  the  Pearson  correlations  for  the  vision  test  and  ACM 
performance  criteria.  One  vision  test  measure,  spot  detection  threshold 
stress  response  time  (SPOT-SRT),  was  significantly  related  to  the  OAG  (r  ■ 
-.22),  The  negative  correlation  indicates  that  longer  spot  detection 
response  times  (i.e.,  slower  performance)  were  associated  with  lower 
(poorer)  OAGs,  Vision  test  scores  were  not  significantly  related  to 
situational  awareness,  VXD  kill,  or  engaged  kill  criteria.  Contrast 
sensitivity  measures  at  spatial  frequencies  (CS  3.0  cycles/degree,  r  -  -.24; 
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CS  1L.4  cycLes/degree,  jr  •  -.26;  «nd  CS  22,8  cycles/degree,  £  ■  -.25)  were 
signlflcaatly  related  to  the  mean  time- to-firjt-Kill  criterion.  In 
addition »  a  high  contrast  acuity  thresholl.d  mean  measure  (ACHI-TM)  was 
significantly  related  to  mean  time  to  first  kill  <£  •>  -.21).  Generally, 
for  the  five  contrast  sensitivity  spatial  frequencies  and  the  high  contrast 
acuity  threshold  test,  poorer  vision  test  scores  were  associated  with 
shorter  mean  times  to  first  kills  (l.e.,  better  ACM  performance).  The 
consistent  direction  and  magnitude  of  the  correlations  between  contrast 
sensitivity  and  mean  time  to  first  kill  Indicate  a  reliable  effect,  although 
not  in  the  expected  direction*  We  expected  better  vision  test  scores  to  be 
associated  with  better  ACM  performance,  however,  mean  time  to  first  kill  was 
Inversely  related  to  age,  Jet  hours,  total  flight  hours,  and  TACTS  hours,  as 
shown  In  Table  8.  Older  and  more  experienced  pilots  achieved  shorter  mean 
times  to  first  kill.  The  literature  Indicates  a  progressive  general 
deterioration  In  visual  functions  with  age.  In  particular,  visual  acuity 
and  contrast  sensitivity  decrease  significantly  with  age  (1C, 11). 

TABLE  7.  Pearson  Correlations  for  the  Vision  Test  and  ACM  Performance  Criteria 
(n  varies). 


Vision 

tsst 


Hsan  time  Mtmber 

Situational  to  first  Visual  tally  VID  times 
auaisoess  kill  mean  range  mean  range  killed 


Kill 

VXD  Engaged  dlffstencs 
kills  kills  scons 


Aaa*TM 

(89) 

.05 

.15 

-.21* 

-.06 

-.04 

-.09 

.14 

2 

• 

1 

.00 

ACHI-SBI 

<88) 

-.08 

,10 

.04 

.19 

.15 

.12 

-.09 

.04 

-.00 

AOIMM 

(89) 

.07 

.12 

-.13 

-.22* 

-.07 

-.13 

-.00 

-.09 

-.01 

ACW-SXS 

(88) 

-.04 

.05 

.08 

.21* 

.18 

.05 

.01 

.18 

.11 

OAR-TM 

(89) 

-.02 

.04 

-.13 

-.26** 

-.12 

-.22* 

.03 

-.15 

-.06 

GUR-^ 

(88) 

-.15 

-.13 

.07 

,12 

,05 

.07 

.03 

.12 

.08 

GURV-TM 

(61) 

.16 

.13 

-.12 

-.14 

-.03 

-.24 

-.00 

.07 

,06 

GUW-SBX 

<61) 

-.05 

1 

• 

s 

.11 

.26* 

,34** 

.10 

-.07 

-.05 

.03 

SPOMM 

(89) 

-.06 

.06 

-.16 

-.22* 

-.03 

-.06 

.10 

-.09 

-.12 

SPOT-SRT 

(88) 

-.22* 

-.09 

.02 

.12 

.03 

.12 

.10 

-.07 

-.05 

FXPIMM 

(63) 

-.U 

-.07 

-.08 

-.02 

.03 

-.08 

.08 

-.07 

-.17 

RtFM-SKT 

(63) 

.01 

-.09 

-.11 

.25* 

.37** 

.16 

.13 

-.05 

.10 

FXNF-TM 

(63) 

.01 

.21 

-.23 

-.05 

-.13 

-.05 

-.08 

-.18 

-.16 

FXMF-SRT 

(63) 

-.04 

-.08 

.05 

.13 

,35** 

.06 

1 

• 

o 

-.06 

.07 

CS  0,5 

(71) 

.04 

.02 

-.22 

.U 

.24* 

.12 

.00 

-.03 

.01 

CS  1.0 

(71) 

-.09 

-.09 

-.17 

.14 

.19 

.13 

-.01 

-.16 

-.15 

CS  3.0 

(71) 

-.06 

-.03 

-.24* 

.08 

.32** 

.04 

-.03 

-.18 

-.14 

CS  6.0 

(71) 

-.06 

-.05 

-.22 

.05 

.11 

.02 

,03 

-.18 

-.15 

CS  11.4 

(71) 

-.09 

-.01 

-.26* 

.04 

.12 

.01 

.04 

-.22 

-.12 

CS  22.8 

(71) 

-.08 

-.12 

-.25* 

o 

• 

1 

-.06 

-.04 

.11 

-.21 

-.24* 

*  £  <  ,05 

**  £  <  ,01 

Mote:  See  text  for  vision  test  Identification, 
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Our  results  suggest  that  experleace  Is  mediating  the  Inverse  relation 
between  contrast  sensitivity  and  acuity,  and  mean  time  to  j^irst  kill.  That 
is,  pilots  who  obtained  shorter  mean  times  to  first  kills  had  poorer 
contrast  sensitivity  and  acuity*  They  also  were  more  experienced  (and 
older)  and  apparently  used  this  experience  to  achieve  faster  kills  in  ACM. 

A  contrast  sensitivity  measure  (CS  22.8  cycles/degree}  was  significantly 
related  to  the  klll~dif ference  score  ~  .24).  The  contrast  sensitivity 

test  score  is  a  negative  number  score.  A  greater  negative  number  indicates 
better  performance*  Thus,  a  negative  correlation  means  that  better  contrast 
sensitivity  is  associated  with  a  greater  kill-difference  score.  The 
direction  of  the  relationship  between  contrast  sensitivity  and  the  kill* 
.difference  score  is  opposite  that  between  contrast  sensitivity  and  mean  time 
to  first  kill*  We  have  indicated  previously  tliat  age  may  be  a  possible 
mediating  variable  between  contrast  sensitivity  and  mean  time  to  first  kill, 
and  may  account  for  the  finding  that  poor  contrast  sensitivity  is  associated 
with  better  ACM  performance*  Now,  results  are  presented  that  do  not  support 
this  interpretation,  i.e*,  better  contrast  sensitivity  was  associated  with  an 
improved  kill-difference  score.  However,  as  shown  in  Table  8,  the  age  and 
experiencs  variables  were  related  to  mean  time  to  kill  but  generally  not  to 
the  kill-difference  score.  Thus,  our  data  may  not  support  an  experience 
effect  mediating  the  relationship  between  contrast  sensitivity  and  the  kill- 
difference  score,  hut  does  regarding  mean  time  to  first  kill. 

As  expected,  the  vision  test  scores  were  related  more  to  the  highly 
vision-dependent  components  of  ACM,  e.g.,  visual  tally  range  and  VID  range 
than  to  other  criteria.  Both  low  contrast  acuity  threshold  mean  (ACLO-TM) 
and  low  contrast  acuity  threshold  stressed  response  time  mean  (ACLO-SRT) 
correlated  significantly  with  visual  tally  range  <£  -.22  and  jr  .21, 

respectively).  Low  contrast  acuity  with  glare  threshold  mean  <GLAR-TM) 
correlated  significantly  with  visual  tally  range  as  well  -.26).  Spot 

detection- threshold  mean  (SPOT-TM)  (jr  -  -.22)  alsr  -  ‘>rrela'ted  significantly 
with  visual  tally  range. 

Low  contrast  acuity  (with  glare  and  visor)  threshold  stress  response 
time  mean  (GLRV-SRT)  was  significantly  correlated  with  both  visual  tally 
range  (jr  ■■  .26,  and  VXD  range  (£  ■■  .34).  Also,  an  accommodative  flexibility 
(far  to  near)  threshold  stress  response  time  measure  (FXFN-SRT)  was 
significantly  correlated  with  both  visual  tally  range  ■  *23),  and  VXD 
range  <£  *37).  The  accommodative  flexibility  (near  to  far)  threshold 

stress  response  time  measure  (FXNF-SRT)  was  significantly  correlated  with  VXD 
range  (£  ••  .35)  but  not  visual  tally  range*  Xn  every  case,  the  significant 
correlations  between  vision  tests  and  visual  tally/VXD  range  were  in  the 
expected  direction.  For  each  of  the  significant  correlations  reported  above, 
the  threshold  mean  (TM)  values  are  negatively  correlated  with  visual  tally 
and  VXD  range,  and  the  threshold  stress  response  time  (SRT)  measures  were 
positively  correlated  with  the  visual  tally  or  VXD  range  measures.  These 
inverse  relationships  between  the  TM  and  SRT  measures  have  been  reported 
elsewhere  (12,13)  and  seem  to  be  reliable  effects.  Apparently,  aviators  with 
a  low  threshold  responue  mean  score  (better  vision  performance)  achieve 
visual  tally  and  VXD  at  longor  ranges.  They  also  have  longer  threshold 
stress  response  times,  thus  producing  negative  correlations  between  SRT 
measures  and  visual  tally  and  VXD  ranges.  Xt  is  uncertain  why  this  occurs. 
One  hypothesis  is  that  better  performers  on  the  threshold  mean  vision  tests 
are  more  deliberate,  and  hence  slower,  in  making  responses. 


other  vlsuiil  tests  that  correlated  significantly  with  VXD  range  (but 
not  visual  tally  range)  were  contrast  sensitivity  at  3.0  cycles/degree  (jr 
>  .32}  and  0.5  cycles/degree  <£  >■  ,24).  These  Matter  correlations  were  loot 
in  the  expected  direction  however.  Ue  anticipated  that  better  contrast 
sensitivity  (a  more  negative  number)  would  be  associated  with  a  longer  (not 
shorter)  VXD  range »  thus  producing  a  negative  rather  than  a  positive 
correlation. 

An  additional  significant  correlation  was  found  between  the  number-of- 
tlmes-killed  score  and  low  contrast  acuity  with  glare  threshold  mean  GLAR-TM 
(jf  ■■  '■.22).  However,  the  negative  correlations  indicated  that  better 
performers  oh  this  vision  test  were  killed  more  often  in  simulated  ACMl 
Again,  age  or  experience  may  be  affecting  these  correlations  since  the  vision 
of  older  pilots  may  be  more  susceptible  to  the  effects  of  glare.  Older 
pilots,  however,  may  use  their  experience  to  an  advantage  in  ACM. 

In  summary,  the  results  indicated  that  vision  tests  scores  were  related 
to  components  of  ACM  performance  that  were  associated  with  highly  vision 
dependent  tasks.  This  should  not  be  too  surprising.  The  vision  test  battery 
was  specifically  designed  to  identify  critical  visual  functions  predictive  of 
succece  in  ACM,  particularly,  the  range  of  initial  target  detection  and 
identification.  Thus,  we  did  not  expect  the  NAMRL  vision  tests  to  correlate 
with  ell  aspects  of  ACM  performance.  For  example,  VXD  kills  are  accomplished 
on  the  initial  pees  where  performance  may  reaaonably  be  dependent  on  the 
range  of  luitiel  visual  ecquieition.  In  contrshit,  an  engaged  kill  occurs 
during  ths  portion  of  ACM  when  each  pilot  trios  to  maneuver  behind  the  other 
to  fire.  Engaged  kills  may  require  fewer  visual  skills  and  are  affected 
comperetively  mote  by  psychomotor  skills,  the  skills  of  the  adversary,  and 
pettlculerly  tactics  associated  with  weapon  system  employment. 

Table  8  presents  corrsletions  between  ACM  performance  criteria  and 
measures  of  experience—speclficelly,  age,  jet  hours,  total  Jet  hours,  TACTS 
hours,  end  toul  ACM  flight  hours.  Of  45  correlations,  8  (18%)  were 
•ignif leant  at  the  .05  or  .01  level  of  confidence.  The  one  criterion 
consistently  related  to  age  or  flight  experience  measures  was  the  mean  time- 
to-f Irrt-kill  score,  which  produced  eignificent  correlations  with  age  (jf  ■  - 
-.22).  In  each  case,  greeter  age  or  more  flight  experience  was  associated 
with  shorter  meeu  time-to-first-kill  scores  (better  performance).  Visual  XD 
performance  was  significantly  related  to  jet  hours  (^  .24)  aud  total  ACM 

flight  hours  (^  “  .21).  The  total  number  of  ACM  flight  hours  was  the  only 
experience  measure  slguif icently  related  to  the  OAG  (r  .23)  and  the  kill 
difference  score  (^  .21),  There  were  no  significan't  correlationa  between 

age  or  flight  experience  measures  and  situational  awareness,  visual  tally 
range,  VID  range,  number  of  times  killed,  and  the  number  of  engaged  kills. 
Further,  only  one  experience  measure,  total  ACM  hours,  was  significantly 
related  to  the  OAG,  VID  kills,  and  the  kill-difference  score.  These 
correlations  were  generally  quite  small,  accounting  for  only  4  or  5%  of  the 
variance  with  each  criterion. 


TABLE  8,  Ptarsot;  Correlations  for  Experience  Measures  and  ACM  Performance  (n  varies). 


ACM  performance 
criteria 

Age 

n-89 

Jet 

hours 

£“88 

Total 

hours 

£-88 

TACTS 

hours 

£-85 

Total 

ACM 

£-85 

OAG 

.10 

.20 

.17 

.13 

.23* 

Situational  awareness 

.04 

.08 

.06 

-.02 

.08 

Mean  time-to-first-kill 

-.34** 

-.36** 

-.32** 

-.22* 

-.16 

Visual  tally  mean  range 

,05 

-.01 

-.04 

.06 

-.01 

Visual  ID  mean  range 

-.02 

-.06 

-.11 

-.08 

.06 

Number  of  times  killed 

-.06 

-.05 

.03 

.14 

-.02 

VXD  kills 

.09 

.24* 

.17 

.16 

.21* 

Engaged  kills 

.13 

.10 

.13 

-.02 

.13 

Kill-difference  score 

*  £  <  ,05 
**  £  <  .01 

.06 

.10 

.07 

.04 

.21* 

Our  results  indicate  that  ACM  experience  influences  ACM  performance, 
especially  in  achieving  VIU  kills  and  Improved  time- to-* first-kill  scores.  On 
the  other  hand,  these  results  also  suggest  that  experience  is  not  related  to 
situational  awareness,  visual  tally,  VXD  range,  engaged  kill,  or  being  killed 
criteria.  Future  evaluations  of  ACM  TACTS  performance  should  iiiclude 
experience  factors  similar  to  those  examined  here  to  better  understand  the 
relation  of  age  and  experience  to  TACTS  ACM  performance. 

DISGUSSXOH  AMD  COMCLUSIOMS 

Study  I.  HuLtitasIc  Test  end  Narine  Pilot  ACH  Perfornence. 

This  evaluation  was  conducted  to  test  the  feasibility  of  predicting  ACM 
performance  with  perceptual  motor  and  cognitive  multitask  tests.  Eighteen  F-4 
pilots  performed  in  Fleet  Fighter  ACM  Readiness  exercises  and  completed 
automated  performance-based  tests. 

Initial  analyses  Indicated  that  the  overall  ACM  grade  (OAG)  associated 
with  the  VF-43  adversary  squadron's  evaluation  of  ACM  performance  of  F-4 
pilots  was  significantly  and  positively  related  to  offensive  maneuvering, 
situational  awareness,  and  mutual  support  measures.  In  addition,  the  OAG  was 
significantly  but  negatively  related  to  the  objective  TACTS  measures,  mean 
time-to-first-kill,  adversary  squadron  missile  shots,  and  the  number  of  times  a 
pilot  was  killed  in  the  simulated  exercises.  The  negative  correlations  indi¬ 
cated  that  a  higher  ACM  grade  was  associated  with  shorter  times  to  first  kill, 
fewer  adversary  squadron  missile  shots  taken,  and  fewer  times  being  "killed"  in 
the  simulated  exercises.  A  multiple  regression  analysis  indicated  that  four  of 
these  measures,  situational  awareness,  offensive  maneuvering,  number  of  times 
killed,  and  mutual  support,  could  reliably  predict  the  OAG  criterion. 

The  OAG  and  the  four  criterion  measures  were  then  correlated  with  the 
single-  and  multitask  cognitive  and  psychoinotor  test  performance  of  the  F-4 
pilots.  A  derived  composite  kill-difference  score,  based  on  the  total  number 
of  ACM  kills  less  the  number  of  times  a  pilot  was  killed  in  the  TACTS  simulated 


exercisiis,  was  lacludad  la  the  correlational  analyses  as  well. 

A  i)LT  measure  obtained  during  multitask  performance  was  significantly 
related  to  offensive  maneuvering  and  the  kill-difference  composite  score.  A 
DLT  multitask  test  measure  based  on  a  slightly  different  scoring  procedure 
was  significantly  related  to  the  OAG  and  the  offensive  maneuvering  scoria. 

Conclusion:  Multitask  tests  reliably  predicted  ACM  performance  for  a 
small  sample  of  jir-4  pilots. 

Unresolved,  however,  was  a  aeries  of  Important  questions  conceimlng 
the  Fleet  Fighter  ACM  Readiness  Program  evaluation  process:  Are  the 
resulting  grades  reliable  for  Navy  pilots  flying  contemporary  F-14  aircraft? 
What  is  the  relation  of  the  VF-43  grading  process  to  more  objective  TACTS 
ACM  performance  measures  (l.e.,  total  number  of  kills,  VXD  range,  VXD  klllb, 
and  engaged  kills)?  And  which  ACM  measures  are  most  predictive  of  ACM 
performance? 

Staidy  lit  Fleet  Fighter  ACH  Readiness  Program  GraiV.s  as  Criteria. 

Objectives  of  Study  XI  were  to  Identify  criteria  for  the  validation  of 
tests  designed  tc  predict  ACM  performance  and  estimate  the  reliability  of 
readiness  grades  used  to  assess  Navy  pilot  ACM  proficiency. 

An  examination  of  subjective  and  objective  measures  of  the  TACTS  ACM 
competitive  exercise  performance  of  125  naval  aviators  participating  In  Fleet 
Fighter  ACM  Readiness  Program  Exercises  at  NAS  Oceana,  Indicated  that  the 
overall  grade  (OAG)  can  be  reliably  predicted  by  a  relatively  few  measures. 
These  were  a  kill-difference  composite  score  (resulting  from  adding  the 
number  of  TACTS  VXD  and  engaged  kills  and  then  subtracting  the  number  of 
times  a  pilot  was  killed  during  TACTS  competitive  exercises),  slfuatlonal 
awareness,  energy  management,  and  mutual  support  measures* 

A  separate  correlational  analysis  examined  the  reliability  of  the  Fleet 
Fighter  ACH  grading  process.  Subjects  were  randomly  divided  Into  two  groups 
and  the  correlations  between  the  various  performance  measures  and  OAG  were 
examined.  The  resulting  £  values  were  highly  similar.  Xn  addition, 
correlation  values  were  c'omputed  for  eight  different  pilot  subsamples  based 
on  a  total  group  prediction  model  of  OAG  and  the  OAG  actually  obtained.  The 
average  of  the  eight  correlation  values  was  .88.  These  results  suggest  that 
the  Fleet  Fighter  Readiness  grading  process  is  reliable.  Regardless  of  the 
subject  sampling  procedure,  the  model  for  predicting  OAG  provided  highly 
similar  results.  Apparently  the  grading  of  ACM  performance  by  different  VF- 
43  adversary  pilots  was  consistent* 

Conclusion:  Fleet  Fighter  ACM  Readiness  program  grades  are  reliable  and 
suitable  criteria  for  validating  tests  designed  to  predict  F'14  pilot  ACM 
performance. 

Sbidy  lilt  The  Relation  of  Vision  Test,  Experience,  and  ACH  Perforeance. 

The  purpose  of  Study  XXX  was  to  examine  the  relation  between 
experimental  NAMRL  vision  tests  and  ACM  performance  measures  identified  in 
Study  XX.  Contrast  sensitivity  tests  were  significantly  correlated  with  a 
mean  time- to-flrst-klll  score,  and  visual  acuity  and  accomodatlve  flexibility 
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t68t8  were  sigalflcencly  correlated  with  TACTS  vieual  tally-ho  and  VXD  range 
criteria*  Many  of  the  vialon  and  TACTS  performance  correlations  aeeced  to  be 
affected  by  an  age  or  experience  factor.  For  example,  pilots  who  obtained 
shorter  mean  times-to-f irst-kills  had  poorer  visual  contrast  sensitivity  and 
acuity*  but  also  were  more  experienced  (and  older)  and  apparently  used  this 
experience  to  achieve  faster  kills  in  ACM. 

Conclusion:  The  NAMRL  vision  tent  scores  were  related  more  to  the  highly 
vision-dependent  components  of  ACM*  e.g**  visual  tally  range  and  VXD  range. 

Additional  correlations  were  computed  between  ACM  performance  criteria 
and  measures  of  experience— ‘Specifically*  age*  jet  hours*  total  jet  hours* 
TACTS  hours  and  total  ACM  flight  hours.  The  one  criterion  consistently 
related  to  age  or  flight  experience  measures  was  the  mean  time- to-first-kill 
score*  producing  significant  correlations  with  age*  jet  uours*  total  flight 
hours*  and  TACTS  hours*  Xn  each  case*  greater  age  or  more  flight  experience 
was  associated  with  shorter  mean  time-to-first-kill  scores  (better 
performance).  Visual  XD  kill  performance  was  significantly  related  to  jet 
hours  and  total  ACM  flight  hours.  Total  ACM  flight  hours  was  significantly 
related  to  the  OAG*  VXD  kills*  and  the  kill-difference  score.  There  were  no 
significant  correlations  between  age  or  flight  experience  measures  and 
situational  awareness*  visual  tally  range*  VXD  range*  number  of  times  killed 
and  the  number  of  engaged  kills. 

Conclusion:  Experience  in  ACM  influences  performance*  especially  in 
achieving  VXD  kills  and  Improved  tlme-to-flrst-klll  scores.  Experience  in 
ACM  was  not  related  to  situational  awareness*  visual  tally*  VXD  range* 
engaged  kill*  and  being  killed  criteria* 

EECONMENDATIOHS 

Results  (Study  X)  demonstrated  the  feasibility  of  using  automated* 
synthetic,  cognitive,  perceptual*  and  multitask  teats  to  predict  TACTS  F-4 
pilot  ACM  proficiency  and  indicated  (Study  XX)  that  Fleet  Tighter  ACM 
Readiness  Program  grades  are  reliable  criteria  for  validating  tests  designed 
to  predict  ACM  performance. 

To  achieve  the  goal  of  validating  tests  to  aid  In  aircrew  selection  and 
assignment  decisions*  the  following  research  is  needed. 

1*  Synthetic  cognitive*  perceptual*  and  multitask  tests  should  be 
administered  to  a  suitable  sample  of  F-14  pilots  performing  in  Fleet  Fighter 
Readiness  Evaluations  to  replicate  initial  test  results* 

2*  Pilot  experience  data  should  be  included  in  the  above  effort  to 
study  the  relation  of  age  and  experience  to  TACTS  ACM  performance. 

3.  An  analysis  of  additional  OAG  data  would  be  useful  in  assessing  the 
reliability  and  validity  of  mathematical  models  to  predict  Fleet  Fighter  ACM 
grades. 

The  successful  validation  of  synthetic  tests  to  predict  ACM  performance 
would  be  valuable  for  improving  the  quality  and  capabilities  of  fighter 
aircrew  through  their  initial  selection  and  subsequent  assignment  to  training 
pipelines  and  aircraft. 


18 


I 


REFERENCES 


1.  Ciavaralll^  A»  Brlctao<i«  G«  A.»  and  Young,  P.  A.,  Davelopment  and 

Application  of  Parformanca  Criteria  and  Aircrew  Assessment  Methods 
^or  tha  Ale  Combat  Maneuvering  Range  (ACMR),~NANRL  Special  Report 
79-5,  Naval  Aerospace  Medical  Research  Laboratory,  Pensacola,  FL, 
1979, 

2.  Bale,  R,  M, ,  Rlckus,  G,  M.,  and  Ambler,  R.  K. ,  Replacement  Air  Group 

Performance  as  a  Criterion  for  Aviation  Training,  NAMRL  1126, 

Naval  Aerospace  Medical  Research  Laboratory,  Pensacola,  FL,  1973. 

3.  Brictson,  C,  A.,  Burger,  W.  J.  and  Gallagher,  T.  J.,  "Prediction  of 

Pilot  Performance  during  Initial  Carrier  Landing  Qualifications." 
Aerospace  Medicine,  Vol.  43,  pp.  483-487,  1972. 

4.  Rlckus,  G,  M,  and  Berkshire,  J.  R. ,  Peve?opment  of  an  Aviation  Combat 

Criterion,  NAMI-1047,  Naval  Aerospace  Medical  Institute, 

Pensacola,  ?L,  1968. 

5.  Shannon,  R.  H.  and  Uaag,  W.  L.,  The  Prediction  of  Pilot  Performance 

In  the  F-4  Aircraft,  NAMRL-1186,  Naval  Aerospace  Medical  Research 
Laboratory,  Pensacola,  FL,  1973. 

6.  Shannon,  R.  H,  and  Uaag,  U.  L.,  Toward  the  Development  of  a  Criterion 

for  Fleet  Effectiveness  In  the  F-4  Fighter  Community.  NAMRL- 117 3, 
Naval  Aerospace  Medical  Research  Laboratory,  Pensacola,  FL,  1973. 

7.  Shannon,  R,  H.,  Uaag,  W,  L. ,  and  Ferguson,  J,  C,  A  New  Approach  to 

Criterion  Development  In  the  Replacement  Air  Group.  NAMRL- 1158, 
Naval  Aerospace  Medical  Research  Laboratory,  Pensacola,  FL,  1973. 

8.  SAS  User's  Guides  Statistics  (5th  ed,),  SAS  Institute,  Inc.  Cary,  NC, 

1985. 

9.  Commander  Fighter  Wing  One  Instruction  3710, 3£,  Naval  Air  Station 

Oceana,  Virginia  Beach,  VA,  11  Feb  1985. 

10.  Pitts,  D,  G. ,  "The  Effects  of  Aging  on  Selected  Visual  Functions:  Dark 

Adaptation,  Visual  Acuity,  Stereopsis,  and  Brightness  Contrast." 

In  R.  Sekuier,  D,  Kline,  and  K,  Dlsmukes.  (Eds.),  Aging  and  Human 
Visual  Function,  Alan  R.  Lias,  Inc.,  NY,  1.982,  pp.  131-160, 

11.  Sekuier,  R.  and  Owsley,  C.,  "The  Spatial  Vision  of  Older  Humans."  In 

R.  Sekuier,  D.  Kline,  and  K.  Dlsmukes.  (Eds.),  Aging  and  Human 
Visual  Function,  Alan  R,  Liss,  Inc,,  NY,  1982,  pp,  185-202, 


12.  Monaco,  W,  A,  and  Hamilton,  P.  V,,  "Visual  Capabilities  Related  to 

Fighter  Aircrew  Performance  in  the  F-14  and  Adversary  Aircraft." 
Advisory  Group  for  Aerospace  Research  and  Development  Conference 
Proceedings  No.  396,  pp.  38-1  to  38-9,  December  1985. 


19 


Morris,  A.,  Hamilton,  P,  V.,  Morey,  W.  A.,  and  Briggs,  R.  P,,  "Vision 
Test  Battery  Threshold  and  Response  Time  as  Predlctore  of  Air- to 
Air  Visual  Target  Acquisition  In  F»14  and  Adversary  Aircraft." 
Advlaory  Group  for  Aerospace  Research  and  Development  Conference 
Proceedings  No.  396,  pp.  39-1  to  39-8,  December  1985 


APPENDIX  A 


APPENDIX  A 


Tti«  Tactical  Aircrew  Combat  Training  System  (TACTS)  la  one  of  the  most 
technologically  sophisticated  training  systems  in  existence.  The  system  is 
a  computer  based  tracking  and  data  communication  network  that  enables  ACM 
training  and  simulated  weapons  firing  of  actual  aircraft  engaged  in  ACM  in 
real  time.  The  TACTS  system  provides  military  managers  with  visual  flight 
dynamics^  weapons  system  status,  and  weapons  firing  information.  All  data 
(including  the  visual  representation  of  aircraft)  are  recorded  on  magnetic 
tape  for  use  in  the  debriefing  of  both  adversary  and  fighter  aircrews.  The 
need  for  such  a  training  system  became  apparent  during  the  Viet  Nam  conflict 
in  which  pilots  often  failed  to  recognize  when  they  were  in  a  correct  firing 
envelope  for  the  missile  weaponry  of  that  day.  The  TACTS  system  enables  the 
employment  of  both  rear-aspect  and  forward-aspect  missile  weaponry  and 
serves  as  a  means  of  evaluating  the  tactical  use  of  both  missile  and 
aircraft  weapon  systems  in  simulated  air  combat. 
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ABBREVIATED  ACM  PERFORMANCE  MEASURE  DEFINITIONS 

OVERALL  ACM  GRADE  -  a  composite  of  19  subjective  measures  >  (see  page  4 
of  text)  Figure  I). 

ENVIRONMENT  -  use  of  weather  conditions  to  gain  an  advantage  in  ACM. 

START/VID  START  -  position  at  start  of  engagement  when  the  fighter  and 
adversary  aircraft  merge*. 

FIRST  MOVE  >  a  positioning  advantage  the  fighter  tries  to  obtain  Just 
before  the  merge*. 

AGGRESSIVENESS  -  how  aggressively  the  fighter  employs  his  aircraft  weapon 
systems. 

OFFENSIVE  MANEUVERING  -  fighter's  ability  to  optimize  offensive  position 
and  achlevo  missile  shots. 

DEFENSIVE  MANEUVERING  -  fighter's  ability  to  maneuver  while  defensive  and 
avoid  being  shot. 

KEEPING  SIGHT  -  awareness  of  position  of  wingmen  and  adversary  aircraft. 

ENERGY  MANAGEMENT  -  optimizing  airspeed  while  maneuvering. 

MENTAL  PLOT  “  fighter's  mental  picture  of  aircraft  positioning  while 
engaged. 

SITUATIONAL  AWARENESS  -  the  total  of  ACM  performance. 

BUGOUT  *  technique  used  to  disengage  from  ACM  and  arrive  at  a  safe  area. 

WEAPON  EMPLOYMENT  -  radar  use  in  intercept  and  use  of  weapons  while  engaged 
in  ACM. 

VID  TECHNIQUE  •  appropriate  use  of  radar  in  the  intercept. 

VID  COMMUNICATION  •  fighter** to- fighter  and  fighter- to-ground  control)  radar 
intercept  communications. 

UHF  COMMUNICATION  -  fighter  to  fighter  communication  while  engaged. 

GAME  PLAN  -  execution  of  tactical  engagement  plan. 

MUTUAL  SUPPORT  -  fighter's  ability  to  protect  and  support  wingmen. 

DEBRIEF  -  participation  in  the  fighter/adversary  debriefing. 

RECONSTRUCTION  -  ability  to  remember  and  reconstruct  the  ACM  fight. 

NUMBER  OF  KILLS  -  combination  of  measures  23  and  24. 

NUMBER  OF  MISSILES  LAUNCHED  -  Self  explanatory. 

NUMBER  OF  VID  KILLS  -  pre-merge*  Rills.  (These  are  made  prior  to  actual 
ACM)  usually  with  forward  aspect  missiles.) 

NUMBER  OF  ENGAGED  KILLS  -  post-merge*  kills.  (Those  made  during  actual 
ACM)  usually  with  heat  seeking  missiles.) 

MEAN  TIME  FIRST  KILL  -  calculated  from  lO-mile  separation  point  of  fighter 
and  adversary  aircraft. 

NUMBER  OF  RADAR  LOCKS  -  Self  explanatory. 

RADAR  LOCK  MEAN  RANGE  -  mean  range  at  which  radar  lock  obtained. 

VISUAL  TALLY-HO  MEAN  RANGE-  mean  range  of  initial  sighting  of  adversary 
aircraft  during  intercepts. 

VID  MEAN  RANGE  -  mean  range  of  adversary  aircraft  identifications)  l.e.) 
"A-4." 

NUMBER  OF  TIMES  KILLED  -  Self  explanatory. 

KILL  DIFFERENCE  SCORE  -  measure  21  minus  30  -  a  composite  score. 

ENGAGED  KILL  DIFFERENCE  SCORE  -  measure  24  minus  30  -  a  composite  score. 


*  Merge  point:  The  point  at  which  the  fighter  and  adversary  aircraft  first 
pass  during  the  intercept. 
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■yAnt.g.  c-L.  Correlation  Matrix  of  TACTS  Subjective  and  Objec 


02.  Environeent  +. 

03.  Start /VID  start  +.404  ■ 
04.  First  Hove  +.119' 

05.  flqgressiveness  +.449  ■ 
06.  Qffensiv*  aan.  +.529 
07.  Defensive  nan.  +.393 
08.  Keeping  sight  +.350 
09.  Energy  eanageeent  +.371 

10.  Hei^al  plot  -.106 

11.  Situational  aeare  +.697 

12.  Bugout  +.112 

11  Ueapon  eeployeent  +.296 

14.  VID  technique  +.302 

15.  VID  coaik  -.078 

16.  UHF  cam.  +.115 

17.  Saee  plan  +.322 

18.  Mutual  support  +.435 

19.  Debrief  +.110 

20.  Reconstruction  +. 162 

21.  Tptal  TACTS  kills  +.648 

22.  Missiles  launched  +.5^ 
21  No.  VID  kills  +.391 

24.  No.enaagMt  kills  +.572 

25.  Tiee  first  kill  -.024 

26.  No.  radar  locks  +.235 

27.  Radar  lock  range  -.040 

28.  Eyetally  x  range  +.118 

31.  Kill  dif. score  +.757 

32.  Engaged  killed  +.704 

difference  score 


-.106  -  3 
-.012  -.087  -  4 
-.040  +.148  +.074  -5 
-.012  +.121  -.066  +.300 
-.105  +.277  -:068  +.280  + 
-.038  +.110  -.019  -.006  + 
+.006  +.194  +.171  -.026  + 
-.061  +.048  -.051  -.055  - 
+.014  +.209  +.004  +.319  + 
-.116  +.110  -.156  -.181  - 
-.085  +.102  -.093  -.160  * 
-.124  +.132  +.182  +.202  • 
+.123  -.154  +.013  -.039  - 
+.062  +.214  -.105  +.145  ^ 
+.059  -.018  +.016  +.159  H 
+.142  -.066  +.243  +.191  ^ 
-.036  +.037  -.008  +.290  ^ 
+.037  -.123 '+.235  +.138  ■ 
+.131  +.253  -.024  +.265  ^ 

+.148  +.283  +.086  +.284  ^ 

+.078  +.181  -.145  +.132  ^ 

+.116  +.201  +.086  +.257  ■ 

+.089  -.058  +.004  +.003  ■ 

i  +.103  +.171  -.022  -.101  ■ 

I  +.030  +.001  +.080  -.182  ■ 

1  +.134  +.180  -.222  -.054 

,  +.044  +.185  -.051  -.015  ■ 

I  +.032  +.199  -.075  -.210 

’  +.107  +.285  +.005  +.314 

I  +.084  +.262  +.107  +.305 


-  7 

-.075  -  a 
+.103  +.057  -9 
+.084  -.046  +.210  -10 
+.235  +.256  +.056  -.155  - 
-.016  +.064  -.041  -.054  +. 
+.031  +.202  +.045  -.139  +. 
+.139  +.134  +.153  +.066  +. 
-.090  +.099  -.019  +.038  -. 
+.155  -.009  -.008  -.025  +. 
-.123  +.053  +.145  -.075  +. 

:  +.108  -.093  +.034  -.330  +. 

,  -.066  -.105  +.009  -.053  +.i 
1  +.068  -.101  +.066  +.092  +.i 
:  +.132  +.278  +.242  -.113  +. 

I  +.111  +.255  +.271  -.060  +. 
h  +.072  +.135  +.201  -.021  +. 

.  +.122  +.271  +.169  -.137  +. 
i  -.121  +.166  +.052  +.0O1  -. 
►  +.074  +.227  +.197  -.064  +. 
i  -.074  +.139  +.002  -.161  -. 
)  +.125  +.053  +.055  -.052  +. 
)  +.021  +.119  +.133  +.015  +. 
5  -.282  -.192  -.219  -.184  -. 
}  +.220  +.312  +.292  -.037  +. 
I  +.224  +.312  +.241  -.039  +. 
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Sttpwls*  tttgr«tsloa  proc«dur«  for  d«pttndttnt  variable  (overall  ACM  grade) 
St«p  Varl«bl«  V8  ent«r«d  Multiple  Ift  "  .91 

^  Square  ■■  0.83 

Adjueted  R  Square  (shrinkage)  **  8  *  *81 


df 

Sum  o£  Squares 

Mean  Square 

F 

£ 

Regression 

6 

0.2658 

0.0443 

98.57 

0.0001 

Error 

118 

0.0530 

0.0004 

Total 

124 

0.3189 

B  Value 

Standard  Error 

Type  11  SS 

F 

£ 

Intercept  1.1111 

V3 

0,0760 

0.0124 

0.0169 

37.62 

0.0001 

V6 

0,0732 

O.OUl 

0.0195 

43.49 

0.0001 

V8 

0.0747 

0.0143 

0.0123 

27.37 

0.0001 

V9 

0.0731 

0.0136 

0.0129 

28.70 

0.0001 

Vll 

0,0869 

0.0087 

0.0445 

99.08 

0.0001 

Via 

0.0669 

0.0077 

0.0336 

74.66 

0.0001 

Summary  of  stepwise  regression  procedure 

for  dependent  variable  VI 

-  OAG 

Number 

Partial 

Model 

Step 

Variable  Entered 

In 

r2 

£2 

1 

Vll  Situational 

awareness 

1 

0.4862 

0.4862 

2 

V6  Offensive  maneuvering 

2 

0.1132 

0.5995 

3 

V18  Mutual  support 

3 

0.0747 

0.6742 

4 

V3  SUrt/VlD  sUrt 

4 

0.0777 

0.7519 

5 

V9  Energy  management 

5 

0.0432 

0.7951 

6 

V8  Keeping  sight/ lookout 

6 

0.0386 

0.8337 

kM  wr«.w.-n4t'f^-v  r\ii  r\‘M:j\Mi/\'M:{\M  n.M-»-W^i£VVi 


TABLE  D-2.  Objective  Measure  Forward  Selection »  Analysis  of  Variance, 
Coefficients,  jF  Values,  and  Model  Summary  Statistics* 


Stepwise  regression  procedures  for  dependent  variable  (overall  ACM  grad^ 
Step  4,  Variable  V25  entered  Multiple  IR  »  *79 

^  Square  ■  0,63 

Adjusted  ^  Square  (shrinkage)  *  R  "  *61 


df 

Sum  of  Squares 

Mean  Square 

L 

£ 

Regression 

4 

0.2008 

0.0502 

51.02 

0.0001 

Error 

120 

0.1181 

0.0010 

ToUl 

124 

0,3189 

B  Value 

Standard  Error 

Type  11  SS 

P 

£ 

Intercept 

1,9973  .. 

V23 

0.0075 

0.0020 

0.0134 

13.62 

0.0003 

V24 

0.0131 

0.0016 

0.0671 

68,17 

0.0001 

V25 

-0,0003 

0.0001 

0,0044 

4,44 

0.0371 

V30 

-0.0205 

0.0026 

0.0624 

63.40 

0.0001 

Bounds  on  condition  number t  I.5B55,  42.7055 

Summary  of  . 

stepwise  regression  procedure 

for  dependent  variable  VI 

-  OAG 

Number 

Partial 

Mouel 

Step 

Variable  Entered  In 

r2 

r2 

1  V24  Engaged  kills 

1  1 

0.3273 

0.3273 

2  V30 

Number  times 

killed  2 

0.1919 

0,5192 

3  V23 

VXD  kills 

3 

0.0968 

0.6160 

4  V25 

Mean  time  to 

first  kill  4 

0.0137 

0.6297 

TABLE  D«3.  Subjactiv«  and  Objective  Measure  Forward  SeLectioa»  Analysis  of 
Variance*  coefficients*  Values*  and  Model  Summary  Statistics. 


Stepwise  regression  procedure  for  dependent  variable  (overall  ACM  grade 
SUp  4*  Variable  V18  entered  Multiple  R  -  .89 

^  Square  ■■  0.78 

Adjusted  K  Square  (shrinkage  "  R  "  .78 


Sum  of  Squares 

Mean  Square 

F 

£ 

Regression 

4 

0.2503 

0.0626 

109.39 

0.0001 

Error 

120 

0.0686 

0.0006 

Total 

124 

0.3189 

B  Value 

Standard  Error 

Type  11  SS 

£ 

£ 

Intercept 

1.5826 

Vll 

0.0880 

0.0101 

0.0432 

75.61 

0.0001 

V9 

0.0757 

0.0154 

0.0138 

24.12 

0.0001 

V18 

0.0400 

0.0088 

0.0119 

20.78 

0.0001 

V3l 

0.0067 

0.0008 

0.0358 

62.59 

0.0001 

Bounds  on  condition  number <  1.5474*  41.0336 


Summary  of  stepwise  regression  procedure  for  dependent  variable  VI  -  OAG 

Step 

as^  -esmismmi 

Number 

Partial 

Model 

Variable  Entered 

In 

r2 

r2 

1 

V31  Kill  difference  score 

1 

0.5724 

0.5724 

2 

Vll  Situational  awareness 

2 

0. 1356 

0.7080 

3 

V9  Energy  management 

3 

0.0395 

0.7475 

4 

V18  Mutual  support 

4 

0.0373 

0.7848 
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